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Abstract 

A general method of coding over expansions is proposed, which allows one to reduce the highly non-trivial 
problem of coding over continuous channels to a much simpler discrete ones. More specifically, the focus is on 
. the additive exponential noise (AEN) channel, for which the (binary) expansion of the (exponential) noise random 

variable is considered. It is shown that each of the random variables in the expansion corresponds to independent 
Bernoulli random variables. Thus, each of the expansion levels (of the underlying channel) corresponds to a binary 
symmetric channel (BSC), and the coding problem is reduced to coding over these parallel channels while satisfying 
the channel input constraint. This optimization formulation is stated as the achievable rate result, for which a specific 



o 

■ choice of input distribution is shown to achieve a rate which is arbitrarily close to the channel capacity in the high 

SNR regime. Remarkably, the scheme allows for low-complexity capacity-achieving codes for AEN channels, using 
the codes that are originally designed for BSCs. Extensions to different channel models and applications to other 
coding problems are discussed. 



OO 



^; I. Introduction 

In this work, we propose the method of constructing (binary) expansions of discrete-time continuous 
alphabet channels, and coding over the resulting set of parallel channels. We apply this coding over 
^ ■ expansions method to additive exponential noise (AEN) channels, where the signal and noise terms 
'. constructing the channel output are represented with their corresponding binary digits. Focusing on 
[ the additive exponential noise component, we show that the binary expansion of the noise consists of 
' I • independent Bernoulli distributed random variables at each level. (The mean of each random variable is a 
^ . function of their level number in the expansion and the mean of the underlying exponential noise.) This 
fSj [ way, thanks to the expansion technique, we show that continuous alphabet channels can be considered as 
^ ; a set of parallel binary symmetric channels (BSCs). 

> ■ Instead of coding for every level in the expansion, we consider signaling over a finite number of levels, 
^ . and we resolve the problem arising from carryovers either by considering them as noise, or by decoding 
^ ] them the least significant bit onwards to the most significant bit. For each case, we state the corresponding 
achievable rate as an optimization problem, where the rate is maximized over the choices of the Bernoulli 
distributions for the signal transmission over each level with the constraint that that the combined random 
variables satisfy the channel input constraint. Then, utilizing an approximately optimal input distribution, 
we show that one can celebrate the achievability of an e-gap to capacity result in the high SNR regime. 
This method together with capacity-achieving low-complexity codes (such as polar coding) allows one to 
achieve the capacity of the AEN in the high SNR regime. 

The additive exponential noise (AEN) channel is of particular interest as it models worst-case noise 
given a mean and a non-negativity constraint on noise [T|- In addition, the AEN model naturally arises in 
non-coherent communication settings, and in optical communication scenarios. (We refer to ^ and |l2l 
for an extensive discussion on the AEN channel.) Verdu derived the optimal input distribution and the 
capacity of the AEN channel in 01. Martinez, on the other hand, proposed the pulse energy modulation 
scheme, which can be seen as a generalization of amplitude modulation for the Gaussian channels. In 
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this scheme, the constellation symbols are chosen as c{i — 1)', for i = 1, - ■ ■ , 2^^ with a constant c, and 
it is shown that the information rates obtained from this constellation can achieve an energy (SNR) loss 
of 0.76 dB (with the best choice of / = i(l + v^)) compared to the capacity in the high SNR regime. 
Another constellation technique for this coded modulation approach is recently considered in (Si, where 
it is shown that log constellations are designed such that the real line is divided into (2M — 1) equally 
probable intervals. M of the centroids of these intervals are chosen as constellation points, and, by a 
numerical computation of the mutual information, it is shown that these constellations can achieve within 
a 0.12 dB SNR gap in the high SNR regime. Our approach, which achieves arbitrarily close to the capacity 
of the channel, outperforms these previously proposed modulation techniques. 

The rest of the paper is organized as follows. The next section describes the AEN channel model. 
In Section [nil we present the key Lemma which shows that one independent Bernoulli distributed 
random variables occur in the binary expansion of an exponential random variable. Armed with this 
result. Section |IV] develops the expansion modulation technique, where we state the main results of the 
paper. Numerical results are provided in Section IIV-C[ and the paper concludes with a discussion section 
(Section jV]). 

IL Channel model and background 
We consider the additive exponential noise (AEN) channel given by 

Y, = X, + N„i = l,--- ,n, (1) 

where Ni are independently and identically distributed according to the additive noise with an exponential 
density with mean E^; i.e., omitting the index i, the noise has the following density: 

1 „^ 

Pr{iV = n} = —e ^iVM(n), (2) 
En 

where u{n) = 1 for n > and u{n) = otherwise. 

The transmitter conveys one of the messages, m, which is uniformly distributed in Ai, i.e., the random 
message m E Ai = {1, ■ ■ ■ , 2"^}; and it does so by mapping the message to the channel input using the 
encoding function /(■) : Ai — t- X^, where X"(m) = /(m), under the constraints that X = 'St and 



1 " 

-E[J2x^]<Ex, (3) 



1=1 



where Ex is the maximum average energy. 

The decoder uses the decoding function g{-) to map its channel observations to an estimate of the 
message. Specifically, g(-) : 3^" — Ai, where the estimate is denoted by M = g{Y^). 

The rate R is said to be achievable, if the average probability of error defined by 

Pe = Yl Pr{^(F") ^ m\m is sent. } (4) 

' ' m.eM 

can be made small for large n. The capacity of AEN is denoted by C, which is the maximum achievable 
rate R. 

The capacity of AEN is given by (Xl, where 

C = log(l + SNR), (5) 

where SNR = and the capacity achieving input distribution is given by 

/ Ex En \ 

= [ {Ex + ENy '^-^'^'' + {Ex + EN f ''^ ) 
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where 6{x) = 1 if x = 0, and otherwise. Note that this is the p*{x) = argmax/(X; Y), where p{y\x) 

is given by the AEN channel ([T])- 

Surprisingly, while the capacity achieving input distribution for the additive white Gaussian noise 
(AWGN) channel is Gaussian, here the optimal input distribution is not exponentially distributed. However, 
we observe that in the high SNR regime, the optimal distribution gets closer to an exponential distribution 
with mean Ex + Em- 



III. Exponential Distribution: Binary Expansion 

We show the following lemma, which allows us to have independent Bernoulli random variables in the 
binary expansion of an exponential random variable. 

Lemma 1: Let Bi's, be independent Bernoulli random variables with parameters given by pi, i.e., 
Pr{B; = 1} = pi and Fr{Bi = 0} = 1 — p;, and consider the random variable defined by 



B= 2'5z 



Then, the choice of 



(=— oo 



1 



Pi 



1 + e^2' ' 

implies that the random variable B is exponentially distributed with mean A^^, i.e., 

Pr{B = b} = Xe'^''u{b). 



(7) 



(8) 



(9) 



Proof: The proof follows by extending the one given in flU. We show the result by calculating the 
moment generating function of B. Using the assumption that {-B/}/gz are mutually independent, we have 



Ms(t) = E[e 



tB} 



Note that for any / G Z, 



E 



E 



O l — ~QO 



H E e*2'^' 



(10) 



l=—oo 



1 + e^2' 
l + e(t-A)2- 

1 + 

Then, using the fact that for any constant a G M, 



+ 1 



1 + e 



A2' 



gt2 _^ gA2 

1 + e^2' 



1=0 



1-e 



2"+1q 



1 - e° 



(11) 



(12) 



we can obtain the following for t < A, 



1=0 



n— i-cxD -'- 1 -|- e~ 
i=0 

1 - e"^ 



1 - e*-^ 

nJ^ 1 _ e*-^ 1 - e-^2"+i 
1 - e-^ 



1 - e 



t-x ■ 



(13) 
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And, similarly, for the negative part, we have 



l=—n 



which further implies that 



1 _ e(t-A)2-" 1 _ e-A 



A(l-e-) ^^3^ 



(A-«)(l-e-^)- 

Thus, finally for any t < \, multiplying equations \T3\ and [151 we get 

MBit) = (16) 

The observation that this is the moment generation function for an exponentially distributed random 
variable with parameter A concludes the proof. ■ 

IV. Modulation with expansions 

Our proposed coding scheme consists of coding over expansion, referred to as expansion modulation 
(EM), and coding for the resulting parallel binary symmetric channels (BSCs). More specifically, EM 
refers to a modulation over the binary expansion of the channel over levels ranging from —Li to L2, i.e., 

L2 -1/2 

Y,^ Yl 2%= Yl '^KX^,l + N,,l), (17) 

l=-Li l=-Li 

where the expansions of the signal and noise are given by 

X,^ Yl and N,^ Y ^'^M- (18) 

l=-Li l=-Li 

When we take the limit Li, L2 — )■ 00, the channel given by (flTl) corresponds to the one given by ©• 

We propose coding over the expansion levels of this channel. More specifically, the least significant bit 
channel is given by 

Yi^.L, = X,,_L, © Ni,-L,,^ = 1, ■ ■ - iV. (19) 

A capacity achieving BSC code is utilized over this channel with input probability distribution given 
by P-Li- (For example, the polar coding method [|5]|, allows one to construct capacity achieving codes 
for the / = —Li level channel.) Instead of directly using the capacity achieving code design, we use the 
combination of the capacity achieving code and the method of Gallager H to achieve a rate corresponding 
to the one obtained by the mutual information I{Xi; Yi) evaluated with an input distribution Bernoulli 
with parameter p/.(The desired distributions will be made clear in the following part.) 

Noting that the sum is a modulo-2 sum in the above channel, there will be carryovers from this sum 
to the next level, / = —Li + 1. Denoting the carryover seen at level / as Cj /, the remaining channels can 
be represented with with the following 

Yi,i = X,,/ © N,^i, i = l,---n, (20) 

where the effective noise, Ny, is a Bernoulli random variable obtained by the convolution of the noise 
and the carryover qi = Fr-{Ny = 1} = © q = - q) + q(1 - qi) with qi = Fr{Ni^i = 1} and 
Q = Fr{Ci^i = 1}. Here, the carry over probability is given by 

Q = Pi-iqi-i, I e {-Li + 1, ■ ■ ■ , L2}. (21) 
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Due to Lemma [H the noise seen at each level will be described by independent Bernoulli random 
variables, and therefore, our coding scheme will be over the parallel channels given by (|20l) for / = 

-Li, ■ ■ ■ , L2, where qi = Pr{Nij = 1} = ® {pi^iQi-i). 



A. Considering carryovers as noise 

Using a capacity achieving code for BSCs, combined with the Gallager's method, expansion modulation 
readily achieves the following result. 

Theorem 2: Expansion modulation, when implemented with capacity achieving codes for the resulting 
BSCs, achieves the rate given by 

L2 

R, = Y,H{pi®qi)~H{qi), (22) 



for any Li, L2 > 0, where qi = qi® {pi~iqi~i) for / > —Li. To satisfy the energy constraint, pi G [0, 0.5] 
is chosen such that 



-E 



n 



i=l 



(23) 



i=l l=-Li 



The optimization problem stated in the result above is highly non-trivial. However, utilizing the optimal 
input distribution, one can adopt the following approximate distributions in Theorem [2l At a high 
SNR, we observe that the optimal input distribution is approximated by an exponential distribution. Then, 
one can simply choose pi from the binary expansion of the exponential distribution with mean Ex + E^. 
To satisfy the power constraint, we use coding only for ^x+En time, and for the rest we set the 

channel input to 0. (As a second approach, in the numerical results, we compare this choice with the 
one of choosing pi from the binary expansion of the exponential distribution with mean Ex-) The next 
method gives a better rate result with a minimal increase in complexity. 



B. Decoding carryovers 

In the scheme above, let us consider decoding starting from the level / = —Li. The receiver will obtain 
the correct for z = 1, ■ ■ • , n. As the receiver has the knowledge of 1^,-Li for z = 1, ■ ■ ■ , n, it will 

also have the knowledge of the correct noise sequence Ni,^Li for i = 1, ■ ■ ■ , A^. With this knowledge, 
the receiver can directly obtain Ci^^n+i for i = 1, ■ ■ ■ , N, which is the carryover from level / = — Li to 
level / = —Li + 1. Using this carryover sequence in decoding at level / = —Li + 1, the receiver can get 
rid of carryover noise. Thus, the effective channel that the receiver will see can be represented by 

Yu=Xij®Nij, 1 = 1, (24) 

for I = —Li, ■ ■ ■ , L2. We remark that with this decoding strategy the effective channels will no longer be a 
set of independent parallel channels, as decoding in one level affects the channels at higher levels. However, 
if the utilized coding method is strong enough (e.g., if the error probability decays to exponentially 
with n), then this carryover decoding error can be made insignificant by increasing n for a given number 
of levels (here, Li + L2 + 1). We state the rate resulting from this approach. 

Theorem 3: Expansion modulation, by decoding the carryovers, achieves the rate given by 

L2 

R2 = Y,H{pi®qi)-H{qi), (25) 
for any Li, L2 > 0, where pi E [0, 0.5] is chosen to satisfy 

^^[E-^-]=^EE (26) 

i=l i=l l=—Li 
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Compared to the previous case, the optimization problem is simpler here as the rate expression is simply 
the sum of the rates obtained from a set of parallel channels. 

We now show that the proposed scheme achieves the capacity of AEN channel in the high SNR regime 
for a sufficiently high number of levels. Towards this end, we provide a bound for the capacity gap, 
AC = C - (7, where 

SNR ^'^^ 

l=-L 

is the achievable rate given in Theorem [3] with Li = L, L2 = "f + L, and 7 = log(l + SNR), when the 
approximate input distribution discussed above (i.e., exponential with mean Ex + E^) is used. First, we 
obtain the asymptotic behavior of entropy at each level. 

Lemma 4: Entropy of the Bernoulli random variable at level /, H[qi), is bounded by 

H{qi) < ci ■ 2"' for / > 0, (28) 
H{qi) > 1 - C2 ■ 2' for / < 0, (29) 

where Ci and C2 are both constants taking the values Ci = 3 log e and C2 = log e. 
Proof: Note that, 

H{qi) = -qi log g/ - (1 - qi) log(l - qi) 

1 1 e'' e^' 

log — ^ - , , log 



When / < 0, we obtain a lower bound as 

.2' 



H{qi)=\og(l + e'')-j^\oge-2^ 



> log(l + 1) - loge ■ 2 



= l-loge-2^ 

On the other hand, when / > 0, by using the facts that log(l + x) < loge ■ x for any < a; < 1, and 

> 1 + X + x^/2 for any x > 0, we have 

Hiq^) = ^ log (1 + e^') + ^ log (l + e^' 

<^log(2e^')+log(l + e-' 

1 + 2' loge 
1 + e^ 
1 + 2' loge 1 

^ 1 + 1 + 2^ + 2272 ^TT2' ^^'' 

< 2"'21oge + 2-' loge 
= 2-'31oge. 

■ 

This lemma tells us that the tails bounds are exponential. Although better bounds may exist, the 
exponential bound is sufficient for further analysis. Based on the Lemma above, we obtain the following. 

Theorem 5: For any e > 0, there exists an e-dependent constant c = log ^^^j^ jj^^j- if ^ > 2c and 

L > c, then the capacity gap is bounded by AC < e. (The total number of levels is given by 2L + 7 + 1, 
where 7 = log(l + SNR).) 
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Levels 

Fig. 1: Signal and noise probabilities, and rate per level, pi, qi, pi®qi and rate at each level are shown, 
where 7 = 15. The coding scheme with L = 5 covers the significant portion of the rate obtained by using 
all of the parallel channels. As shown in the text, pi is a shifted version of qi. 



Proof: We first observe that 

L+7 L+7 L+7 

J2 [H{pi ® qi) - H{qi)] = [H{Pi ® ft) - H{pi)] + ^ [H{pi) - H{qi)] , 

l=-L l=~L l=-L 

L+7 

> J2 \H{'Pi) - H{qi)] , (30) 

l=-L 

where the last inequality is due to the fact that H{pi ® qi) > H{pi). Observing that 

1 

and adopting Lemma |4l we have 

(1) c < / < 7 - c: H{pi) - H{qi) > 1 - ■ 2'"^ - Ci ■ 2-^ 

(2) -c < / < c: [H{pi) - H{qi)] + [^^(^,+7) - ^^(^+7)] = H{pi) - H{ni+^) > 1 - C2 ■ 2'"^ - ci ■ 2-('+^) 
and 

(3) -L < / < -c and 7 + c < / < 7 + L: H{pi) - H{qi) > 0. 



Pi+i = -^—^ = (Ih (31) 
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Combining these pieces together, we will obtain 

I/+7 7— c c— 1 

l=-L l=c l=-c+l 

> 7 - (ci + C2)2-^ - (ci + C2)2^-^-^ 

>7-81oge-2-^ (32) 
where the last inequality uses the assumption that 7 > 2c. Thus, we obtain a bound for the capacity gap 



2'^ — i 

AC = 7 - 5^ I [H{pi ® gz) - i^(Pz)] + [Hipi) - H{q,)] | 

< 7 - (1 - 2-^) (7 - 81oge ■ 2-") 

< 72-^ + 8 lege - 2"^ 

< 2c- 2~2' + 81oge-2-' 

< 161oge-2-'= 

= e, (33) 

where we used the decreasing property of f{x) = x ■ 2~^ for x > 1, and the assumption that 7 > 2c. 
Fig. [T] helps explicate the key steps of the proof. 



C. Numerical results 

We calculate the rates obtained from the two schemes above (i?i in Theorem [2] and R2 in Theorem [3]) 
with two different input probability distribution choices (denoted by Ci and C2): 

• Ci. Choosing pi from the binary expansion of the exponential distribution with mean Ex + E^. To 
satisfy the power constraint, we use coding only for the fraction of the channel uses (i.e.. Ex / (Ex + 
En) of the time). 

• C2: Choosing pi from the binary expansion of the exponential distribution with mean Ex- 

The first choice closely resembles the optimal distribution given in However, as the unused channels 
vanish in the high SNR regime, we expect that both choices result in the same rate as SNR gets large. 
Numerical results are given in Fig. [21 It is evident from the figure (and from the analysis given in 
Theorem [5]) that the proposed technique, when implemented with sufficiently large number of levels, 
outperforms the SNR gaps previously reported in [2] and |l3l. 



V. Discussion 

We note the foUowings. 

• Expansion coding allows the construction of good channel codes for discrete-time continuous channels 
using good discrete memory less channel codes. For instance, one can utilize binary expansion together 
with polar codes. The underlying code is g-ary polar code, as we need to implement Gallager's method 
in constructing the input distribution p{x) = Ber(pj) for coding over level /. (See, e.g., |I3 for 
details.) In addition, expansion modulation can be implemented over a g-ary expansion of the channel, 
and any good code for the resulting modulo g-sum channel can be used. 

• Avestimehr et al. [8] have introduced the deterministic approximation approach for point-to-point and 
multi-user channels. The basic idea is to construct an approximate channel for which the transmitted 
signals are assumed to be noiseless above the noise level. Using this high SNR approximation, one 
only needs to deal with the interference seen at a particular receiver (in a networked model). Our 
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20 

SNR (dB) 

Fig. 2: Numerical results. Ri. The rate obtained by considering carry over as noise. R2: The rate obtained 
by decoding carry overs at each level. Ci. Choosing pi from the binary expansion of the exponential 
distribution with mean Ex + E^. To satisfy the power constraint, only a fraction of the channel uses (i.e., 
Ex+En time) is utilized. C2: Choosing pi from the binary expansion of the exponential distribution 

with mean Ex- Solid and dotted curves correspond to coding over 41 and 21 number of levels, respectively. 



expansion coding scheme can be seen as a generalization of these deterministic approaches. Here, 
the (effective) noise in the channel is carefully calculated and the system takes advantage of coding 
over the noisy levels at any SNR. This generalized channel approximation approach can be useful 
in reducing the large gaps reported in the previous works. 
• Although our discussion is limited to the AEN channel, where the proposed scheme outperforms 
the previously proposed modulation schemes and performs arbitrarily close to the capacity of the 
AEN channel, the expansion coding refers to a more general framework and is not limited to such 
channels. Towards this end, our ongoing efforts are focused on utilizing the proposed scheme for 
AEN multiple-user channels, and for their AWGN counterparts. 
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Appendix A 
Proof of Lemma |4] 

Recall that, in our construction, qi is the probability that Bernoulli random variable takes value 1 at 
level /. Thus, 

Hiqi) = -qi loggz - (1 - qi) log(l - qi) 

1 1 ^2' ^2' 



log . , 01 - . . 01 log 



1 + e2' ^ 1 + 1 + e2' ^ 1 + e^' ' 
When / < 0, we obtain a lower bound as 

H{q{) = log (l + e'') - lege ■ 2^ 

> log(l + 1) - lege ■ 2' 
= 1- lege -2'. 

On the other hand, when / > 0, by using the facts that log(l + x) < lege ■ x for any < x < 1, and 

> 1 + X + x^/2 for any x > 0, we have 



H{qi) = log (l + e^') + log (l + e 

<^^log (2e^')+log (l + e"^' 

l + 2Moge 
1 + 

1 + 2'loge 1 , 

< \ loa: e 

1 + 1 + 2^ + 2272 1 + 2' ^ 

< 2"'21oge + 2~'loge 
= 2-'31oge. 
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